Papers with generative language models
What do Large Language Models Learn about Scripts? (2022.starsem-1)
Copied to clipboard
| Challenge: | Script Knowledge is important for language understanding but expensive to produce manually and difficult to induce from text due to reporting bias. |
| Approach: | They propose a pipeline-based script induction framework which can generate good quality ESDs for unseen scenarios. |
| Outcome: | The proposed framework produces good quality ESDs for unseen scenarios, but manual evaluation shows there is room for improvement. |
Enhancing Language Model with Unit Test Techniques for Efficient Regular Expression Generation (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent studies have shown that generative language models lack functional correctness, which is a critical aspect of regular expressions. |
| Approach: | They propose a method that takes functional correctness into account and transforms it into a differentiable gradient feedback using policy gradient techniques. |
| Outcome: | The proposed method has been used in a regulatory scenario to ensure that all online content is free from non-compliant elements, thereby significantly reducing the workload of relevant personnel. |
Evaluating and Modeling Attribution for Cross-Lingual Question Answering (2023.emnlp-main)
Copied to clipboard
Benjamin Muller, John Wieting, Jonathan Clark, Tom Kwiatkowski, Sebastian Ruder, Livio Soares, Roee Aharoni, Jonathan Herzig, Xinyi Wang
| Challenge: | Open-retrieval question answering systems are lacking in attribution for cross-lingual question answering . open-research questions are available in 20 languages, but their raw generation often falls short in factuality . |
| Approach: | They are the first to study attribution for cross-lingual question answering . they collect data in 5 languages to assess the attribution level of a state-of-the-art QA system . |
| Outcome: | The proposed approach improves the attribution level of a state-of-the-art cross-lingual QA system. |
MiLe Loss: a New Loss for Mitigating the Bias of Learning Difficulties in Generative Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing generative language models neglect an inherent challenge in text corpus during training, i.e., the imbalance between frequent tokens and infrequent ones. |
| Approach: | They propose a function to mitigate the imbalance between frequent and infrequent tokens . authors propose 'MiLe Loss' function to assess learning difficulty of tokens during training . |
| Outcome: | Experiments show that models with proposed model can improve on downstream benchmarks. |
Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI (2024.naacl-demo)
Copied to clipboard
Elron Bandel, Yotam Perlitz, Elad Venezian, Roni Friedman, Ofir Arviv, Matan Orbach, Shachar Don-Yehiya, Dafna Sheinwald, Ariel Gera, Leshem Choshen, Michal Shmueli-Scheuer, Yoav Katz
| Challenge: | Textual data processing pipelines are tailored to specific datasets, task and model combinations. |
| Approach: | They propose a library for customizable textual data preparation and evaluation tailored to generative language models. |
| Outcome: | Unitxt is a library for customizable textual data preparation and evaluation tailored to generative language models. |
Revealing the Unwritten: Visual Investigation of Beam Search Trees to Address Language Model Prompting Challenges (2025.acl-demo)
Copied to clipboard
Thilo Spinner, Rita Sevastjanova, Rebecca Kehlbeck, Tobias Stähle, Daniel A. Keim, Oliver Deussen, Andreas Spitz, Mennatallah El-Assady
| Challenge: | generative language models have become popular, but comprehending their outputs remains challenging for NLP practitioners and linguistic experts. |
| Approach: | They propose to use a beam search tree to examine model outputs to provide information on runner-up candidates and their corresponding probabilities to address these challenges. |
| Outcome: | The proposed method validates existing results and offers additional insights. |
Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Data (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing text classification methods focus on a fixed label set, but many real-world applications require extending to new fine-grained classes as the number of samples per label increases. |
| Approach: | They propose a problem called coarse-to-fine grained classification that leverages label surface names as the only human guidance. |
| Outcome: | The proposed method outperforms existing methods on two real-world datasets. |
Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning (2020.emnlp-main)
Copied to clipboard
Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena D. Hwang, Ronan Le Bras, Antoine Bosselut, Yejin Choi
| Challenge: | Existing methods for integrating past and future contexts are limited and require manual input. |
| Approach: | They propose an unsupervised decoding algorithm that incorporates past and future contexts using off-the-shelf, left-to-right language models and no supervision. |
| Outcome: | The proposed method outperforms unsupervised methods on abductive and counterfactual reasoning tasks. |
LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning (2023.acl-short)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have been limited when fine-tuned on small datasets. |
| Approach: | They propose to add contrastive learning to prompt-based fine-tuning to improve model performance. |
| Outcome: | The proposed approach outperforms other methods on multiple text classification benchmarks. |
Incorporating Stylistic Lexical Preferences in Generative Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in language modeling have resulted in powerful generation models, but their style is implicitly dependent on the training data and cannot emulate a specific target style. |
| Approach: | They propose an approach to induce certain target-author attributes by incorporating continuous multi-dimensional lexical preferences of an author into generative language models. |
| Outcome: | The proposed model generates text that aligns with a given target author’s lexical style and is competitive with baselines. |
Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models (2026.acl-srw)
Copied to clipboard
| Challenge: | Model calibration is commonly used as a proxy for reliability, but low calibration error does not necessarily imply reliable decision rules. |
| Approach: | They investigate the calibration error gap in generative language models using the TOFU benchmark and attribution-based shortcut detection. |
| Outcome: | The proposed model calibrations achieve low calibration error compared to pretrained models and retain low calibration despite reduced accuracy on the forget split. |
TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in generative language models have enabled machines to generate realistic texts. |
| Approach: | They propose a benchmark environment to test the 'Turing Test' problem for neural text generation methods. |
| Outcome: | The proposed benchmark environment is based on 200K human- or machine-generated samples across 20 labels Human, GPT-1, GTP-2_small, GTT-2_medium, GPG-2_large, GGT-2_PyTorch, GGP-3, GROVER_base, griover_large and GRover_mega. |
Opportunities and Challenges in Neural Dialog Tutoring (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to designing dialog tutors have been challenging . current approaches perform poorly in constrained learning scenarios, authors find . |
| Approach: | They analyze dialog tutoring models using automatic and human evaluations to understand the new opportunities brought by dialog tutors. |
| Outcome: | The proposed models perform poorly in less constrained learning scenarios, the authors show . they find large number of model reasoning errors in 45% of conversations . |
In What Languages are Generative Language Models the Most Formal? Analyzing Formality Distribution across Languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual generative language models (LMs) are fluent in a large variety of languages. |
| Approach: | They analyze formality distributions of XGLM and BLOOM’s predictions in 5 languages and classify 1,200 generations per language as formal, informal, or incohesive. |
| Outcome: | The proposed models generate a significant amount of informal predictions even when prompted with formal text. |
Gradient-Based Language Model Red Teaming (2024.eacl-long)
Copied to clipboard
| Challenge: | generative language models generate unsafe responses by producing adversarial prompts . red teaming is labor-intensive and difficult to scale when done by humans. |
| Approach: | They propose a red teaming method that generates diverse prompts that are likely to cause an LM to generate unsafe responses. |
| Outcome: | The proposed method is more effective at finding prompts that trigger an LM to generate unsafe responses than a strong reinforcement learning-based red teaming approach. |
MAGMA – Multimodal Augmentation of Generative Models through Adapter-based Finetuning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pretraining is becoming the norm in Vision-Language (VL) modeling. |
| Approach: | They propose a method for augmenting generative language models with additional modalities using adapter-based finetuning. |
| Outcome: | The proposed method outperforms Frozen on open-ended generative tasks while maintaining the language model weights. |
On Training Data Influence of GPT Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | generative language models have redefined performance standards across tasks . current research on the influence of training data on autoregressivity remains underexplored . |
| Approach: | They propose a parameterized simulation to assess the impact of training examples on the training dynamics of GPT models. |
| Outcome: | The proposed approach compares existing methods with existing methods across training scenarios in generative language models, spanning tasks across 14 million to 2.8 billion parameters. |
Evaluation Metrics for Headline Generation Using Deep Pre-Trained Embeddings (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent generative language models have shown promise in abstractive summarization tasks. |
| Approach: | They propose to use Fr echet embedding distance and angular embeddable similarity to evaluate the performance of generative language models in abstractive summarization tasks. |
| Outcome: | The proposed metric shows close relation with human judgments and has overall better correlations with them. |
Pinpointing Diffusion Grid Noise to Enhance Aspect Sentiment Quad Prediction (2024.findings-acl)
Copied to clipboard
| Challenge: | Current studies on aspect-based sentiment analysis focus on essential content for model generation, ignoring the incorporation of various noise during training. |
| Approach: | They propose a grid noise-diffusion pinpoint network (GDP) model that incorporates three new modules to tackle generation instability. |
| Outcome: | The proposed model reduces the generation instability of model learning and outputs by incorporating Consistency Likelihood Learning and GDP-FOR. |
PRODIGy: a PROfile-based DIalogue Generation dataset (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing profiles-based dialogue datasets lack explicit profile representations or are difficult to collect. |
| Approach: | They propose a dataset that brings together multiple profiles for each speaker, and then integrates them together to provide a more comprehensive profile dimension set for generative language models. |
| Outcome: | The PRODIGy dataset provides a more comprehensive profile dimension set for each speaker. |
SumCSE: Summary as a transformation for Contrastive Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Sentence embedding models are typically trained using contrastive learning (CL) using human annotations directly or by repurposing other annotated datasets. |
| Approach: | They propose to use generative language models to generate CL data using annotated data. |
| Outcome: | The proposed method outperforms the previous best unsupervised method by 1.8 points and SimCSE, a strong supervised baseline by 0.3 points on the semantic text similarity (STS) benchmark. |
CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | a dataset of Chinese large language models is used to measure societal biases . many studies have shown that LLMs exhibit harmful societal biased outputs despite human data . |
| Approach: | They present a Chinese Bias Benchmark dataset that includes over 100K questions constructed by human experts and generative language models. |
| Outcome: | The proposed dataset covers stereotypes and societal biases in 14 social dimensions related to Chinese culture and values. |
Bayesian Calibration of Win Rate Estimation with LLM Evaluators (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) show the potential of using LLMs as evaluators for text quality evaluation. |
| Approach: | They propose two methods to improve the accuracy of LLM evaluators by Bayesian inference. |
| Outcome: | The proposed methods improve the accuracy of the win rate estimation using LLMs . the proposed methods are based on six datasets covering story generation, summarization, and instruction following tasks . |
Generation-Augmented and Embedding Fusion in Document-Level Event Argument Extraction (2025.coling-main)
Copied to clipboard
| Challenge: | Document-level event argument extraction is a crucial task that aims to extract arguments from the entire document, beyond sentence-level analysis. |
| Approach: | They propose a novel approach to document-level event argument extraction that integrates predefined templates and generative language models into a foundational embedding derived from a classification model. |
| Outcome: | The proposed approach is more effective than baseline models and data-efficient in low-resource scenarios. |
ATG: Benchmarking Automated Theorem Generation for Generative Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing generative language models (LMs) can generate new or reusable theorems, but their ability to generate new theorels is under-explored. |
| Approach: | They propose to use Metamath library to generate new theorems that can be saved as reusable knowledge for future theoretical proving. |
| Outcome: | The proposed benchmark evaluates whether an agent can generate valuable (and possibly brand new) theorems that are applicable for downstream theoretic proving as reusable knowledge. |
Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that large language models can cause harmful, human-like biases against various demographics. |
| Approach: | They propose a causal formulation for bias measurement in generative language models based on a list of desiderata for designing robust bias benchmarks and a bias-measuring procedure to investigate occupational gender bias. |
| Outcome: | The proposed framework is generalizable and can be extended to include other datasets. |
Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models for zero-shot cross-lingual event argument extraction are based on pre-trained generative language models. |
| Approach: | They propose to use pre-trained generative language models to generate sentences that fill in a template with arguments extracted from the input passage. |
| Outcome: | The proposed model outperforms the current state-of-the-art models on zero-shot cross-lingual EAE. |
PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have raised concerns about inference costs, increasing the need for research into model compression. |
| Approach: | They propose a method that utilizes prompt tuning to enable generative language models to transfer student-friendly knowledge. |
| Outcome: | Extensive experiments on instruction-following datasets show that PromptKD achieves state-of-the-art performance while adding only 0.0007% of the teacher’s parameters as prompts. |
ClinicalT5: A Generative Language Model for Clinical Text (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent generative language models like BART and T5 are gaining popularity with their competitive performance on text generation and tasks cast as generative problems. |
| Approach: | They propose to build domain-specific PLMs through fine-tuning or pre-training from scratch over domain corpora. |
| Outcome: | The proposed model outperforms existing models on domain-specific tasks and compares favorably with its close baselines. |
Investigating Memorization of Conspiracy Theories in Text Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies examine conspiracy theories in social media, but they have not evaluated their presence in generative language models. |
| Approach: | They examine the ability of generative language models to generate conspiracy theory text . they highlight the difficulties of this task and discuss the drawbacks . |
| Outcome: | The proposed model can generate conspiracy theories without access to training data. |
Retrieval-Augmented Parsing for Complex Graphs by Exploiting Structure and Uncertainty (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Retrieval augmentation is effective for large graph parsing tasks, but can fail to identify the most informative exemplars . structure-aware and uncertainty-guided adaptive retrieval (SUGAR) exploits two unique sources of information: structural similarity and model uncertainty. |
| Approach: | They propose a structure-aware and uncertainty-guided adaptive retrieval approach that exploits structural similarity and model uncertainty to improve retrieval-augmented parsing for complex graph problems. |
| Outcome: | The proposed method improves retrieval-augmented parsing for graph parsers with large output graphs and non-trivial structure. |
Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignment (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies show that multilingual generative models exhibit a strong language bias toward high-resource languages. |
| Approach: | They propose a cross-lingual alignment framework exploiting pairs of translation sentences to improve cross-linguistic abilities. |
| Outcome: | The proposed framework improves cross-lingual abilities and mitigates performance gap. |
LLM4RE: A Data-centric Feasibility Study for Relation Extraction (2025.coling-main)
Copied to clipboard
| Challenge: | Relation Extraction (RE) is a critical step in information extraction due to its wide-scale applicability for downstream applications such as Knowledge Base creation and Question Answering (QA). |
| Approach: | They propose to conduct the first feasibility analysis to explore the viability of Large Language Models for RE by investigating their robustness to various RE scenarios stemming from data-specific characteristics. |
| Outcome: | The proposed models are robust to various RE scenarios stemming from data-specific characteristics, but their performance is not yet fully understood. |
Distractor Generation Using Generative and Discriminative Capabilities of Transformer-based Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Multiple Choice Questions (MCQs) are used to test language learners' comprehension and knowledge. |
| Approach: | They propose an automatic distractor generation approach which generates correct and incorrect answer options and then discriminates potential correct options from distractors. |
| Outcome: | The proposed approach outperforms previous models on multiple choice questions and reading comprehension questions. |
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing web crawling pipelines are used to collect large corpora raw data, but the main way to collect such data is through manual data extraction. |
| Approach: | They propose to use a web crawler to extract and classify data from a multilingual web corpus and an automated annotation pipeline to improve it. |
| Outcome: | The proposed version of OSCAR could be used to pre-train large generative language models and other applications in Natural Language Processing and Digital Humanities. |
Language models emulate certain cognitive profiles: An investigation of how predictability measures interact with individual differences (2024.findings-acl)
Copied to clipboard
| Challenge: | incorporating cognitive capacities increases predictive power of surprisal and entropy measures on reading data, whereas high performance in the psychometric tests is associated with lower sensitivity to predictability effects. |
| Approach: | They examine the predictive power (PP) of surprisal and entropy estimated from generative language models (LMs) on reading data from individuals who also completed a wide range of psychometric tests. |
| Outcome: | The LMs' predictive power is based on cognitive capacities and high performance in psychometric tests is associated with lower sensitivity to predictability effects. |
Soft Alignment Objectives for Robust Adaptation of Language Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Domain adaptation is a common approach for generative language models, but it is notorious for over-specialization to the target domain, resulting in catastrophic forgetting. |
| Approach: | They propose to build training objectives on a semantic similarity of predicted tokens to the reference and avoid catastrophic forgetting of adaptation by preserving adaptation in-domain quality. |
| Outcome: | The proposed objectives mitigate catastrophic forgetting while preserving the adaptation in-domain quality while reducing computational costs. |
Palette of Language Models: A Solver for Controlled Text Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have revolutionized text generation with their remarkable capabilities. |
| Approach: | They propose to combine a single-attribute model with a discriminative model to achieve a combination strategy that incorporates positive correlation and attribute enhancement. |
| Outcome: | The proposed method is adapted for single-attribute control scenario and achieves surpassing results. |
Track-SQL: Enhancing Generative Language Models with Dual-Extractive Modules for Schema and Context Tracking in Multi-turn Text-to-SQL (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to generative language models struggle to handle the increasing complexity of multi-turn Text-to-SQL tasks. |
| Approach: | They propose a framework which enhances generative language models with dual-extractive modules designed to track schema and contextual changes in multi-turn Text-to-SQL. |
| Outcome: | The proposed framework achieves state-of-the-art performance on SparC and CoSQL datasets and significantly improves execution accuracy in multi-turn interactions by 7.1% and 9.55%. |
CoELM: Construction-Enhanced Language Modeling (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that integrating constructional information can improve the performance of pre-trained language models. |
| Approach: | They propose a construction-Enhanced language model that embeds constructional semantics into language models for natural language generation. |
| Outcome: | The proposed model outperforms existing models on various benchmarks. |
Enhancing Pre-Trained Generative Language Models with Question Attended Span Extraction on Machine Reading Comprehension (2024.emnlp-main)
Copied to clipboard
| Challenge: | Extractive Machine Reading Comprehension (MRC) is a challenging field in the field of Natural Language Processing. |
| Approach: | They propose a Question-Attended Span Extraction module to address the limitations of generative approaches for extractive machine reading comprehension (MRC) . module significantly enhances performance of pre-trained generative language models, enabling them to surpass the extractive capabilities of advanced Large Language Models (LLMs) |
| Outcome: | The QASE module surpasses state-of-the-art models in few-shot settings. |
Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in generative language models have demonstrated their ability to memorize knowledge from documents and recall knowledge to respond to user queries effectively. |
| Approach: | They propose to enable multimodal large language models to memorize and recall images within their parameters. |
| Outcome: | The proposed model performs well even with large-scale image candidate sets. |
Leveraging QA Datasets to Improve Generative Data Augmentation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP have substantially improved the capability of pretrained language models to generate high-quality text. |
| Approach: | They propose to reformulate data generation as context generation for a given question-answer (QA) pair and leverage QA datasets for training context generators. |
| Outcome: | The proposed approach improves performance for few-shot and zero-shot classification datasets on multiple classification dataset. |
Distillation Enhanced Generative Retrieval (2024.findings-acl)
Copied to clipboard
| Challenge: | Generative retrieval is a promising new paradigm in text retrieval that generates identifier strings of relevant passages as the retrieval target. |
| Approach: | They propose a framework that leverages generative language models to enhance generative retrieval by distillation. |
| Outcome: | The proposed framework achieves state-of-the-art performance among the generative retrieval methods. |
Graceful Forgetting in Generative Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained models do not provide all knowledge needed for fine-tuning tasks. |
| Approach: | They propose a framework to achieve graceful forgetting in generative language models by pre-training a model on large-scale correlating datasets. |
| Outcome: | The proposed framework improves the learning plasticity of the target task by selectively discarding irrelevant knowledge. |
TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models (2023.emnlp-main)
Copied to clipboard
Jing Xiong, Jianhao Shen, Ye Yuan, Haiming Wang, Yichun Yin, Zhengying Liu, Lin Li, Zhijiang Guo, Qingxing Cao, Yinya Huang, Chuanyang Zheng, Xiaodan Liang, Ming Zhang, Qun Liu
| Challenge: | Automated theorem proving (ATP) benchmarks focus on symbolic inference but rarely involve understanding complex number combination reasoning. |
| Approach: | They propose a benchmark that requires a model to reduce a trigonometric expression with step-by-step proof and evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms. |
| Outcome: | The proposed benchmark evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms. |
Natural Response Generation for Chinese Reading Comprehension (2023.findings-emnlp)
Copied to clipboard
| Challenge: | MRC models trained on labeled answers are limited in generating human-like responses in real QA scenarios. |
| Approach: | They construct a dataset called Penguin to promote machine reading comprehension . they use 200k training data with fluent, well-informed responses to train models . |
| Outcome: | The proposed dataset is the first benchmark towards natural response generation in Chinese MRC on a relatively large scale. |
KLAAD: Refining Attention Mechanisms to Reduce Societal Bias in Generative Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models exhibit societal biases in their outputs, prompting ethical and societal challenges. |
| Approach: | They propose an attention-based debiasing framework that implicitly aligns attention distributions between stereotypical and anti-stereotypical sentence pairs without directly modifying model weights. |
| Outcome: | The proposed framework improves on BBQ and BOLD benchmarks while maintaining fluency and coherence. |
“Was it “stated” or was it “claimed”?: How linguistic bias affects generative language models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Several studies have identified such linguistic classes of words that occur frequently in natural language text and are bias-inducing by virtue of their framing effects. |
| Approach: | They propose to use linguistic cues to induce subtle biases through implied sentiment and presupposed facts to influence the distribution of the generated text. |
| Outcome: | The proposed models are sensitive to these framing effects, but show that they lead to measurable style and topic differences in the generated text, leading to language that is, on average, more polarised and more skewed towards controversial entities and events. |
Transforming Sequence Tagging Into A Seq2Seq Task (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained, large, generative language models have had great success in a wide range of sequence tagging and structured prediction tasks. |
| Approach: | They propose to use a new format for casting input text sentences and their output labels into the input and target of a Seq2Seq model and introduce it to test their hypothesis. |
| Outcome: | The proposed format shows to be both simpler and more effective and devoid of hallucination. |
Uncertainty-Aware Unlikelihood Learning Improves Generative Aspect Sentiment Quad Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on what to generate but ignore what not to generate . a template-agnostic method boosts original learning and reduces mistakes simultaneously . |
| Approach: | They propose a template-agnostic method to control the token-level generation . they introduce Monte Carlo dropout to understand the built-in uncertainty of pre-trained language models . |
| Outcome: | The proposed method boosts original learning and reduces mistakes simultaneously on four public datasets. |
MedQA-SWE - a Clinical Question & Answer Dataset for Swedish (2024.lrec-main)
Copied to clipboard
| Challenge: | MedQA-SWE is a clinical question & answering dataset in Swedish . it was created from exams aimed at evaluating doctors’ clinical understanding and decision making . |
| Approach: | They propose to create a multiple choice, clinical question & answering (Q&A) dataset in Swedish consisting of 3,180 questions. |
| Outcome: | The proposed dataset includes 3,180 questions and is the first open-source clinical Q&A dataset in Swedish. |
GTA: Gated Toxicity Avoidance for LM Performance Preservation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing Controllable Text Generation methods that generate toxic text can negatively impact the performance of the language model. |
| Approach: | They propose a gated Toxicity Avoidance method that can be applied to any CTG method and evaluate its effectiveness. |
| Outcome: | The proposed method reduces toxicity and preserves performance while preserving the language model's generation performance. |
Assessing French Readability for Adults with Low Literacy: A Global and Local Perspective (2025.emnlp-main)
Copied to clipboard
| Challenge: | illiterate individuals are persons aged 15 years and above who cannot read and write with understanding a short simple statement on their everyday life. |
| Approach: | They propose a novel approach to assess french text readability for adults with low literacy skills using a global and segment-level difficulty scale. |
| Outcome: | The proposed approach addresses both global (full-text) and local (segment-level) difficulty scales. |
Nullspace Disentanglement for Red Teaming Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work mainly leverages crowd workers to construct test cases. |
| Approach: | They propose a black-box approach that exploits the unique properties of the nullspace to disentangle and regulate the crucial success information within test cases. |
| Outcome: | The proposed approach outperforms baseline methods regarding the attack success rate and excels in aspects of diversity and fluency. |
Tell, Don’t Show: Leveraging Language Models’ Abstractive Retellings to Model Literary Themes (2025.findings-acl)
Copied to clipboard
| Challenge: | Literature challenges traditional bag-of-words approaches for topic modeling because narrative language focuses on immersive sensory details instead of abstractive description or exposition. |
| Approach: | They propose a topic modeling approach that prompts generative language models to *tell* what passages *show*, thereby translating narratives’ surface forms into higher-level concepts and themes. |
| Outcome: | The proposed model can translate narratives’ surface forms into higher-level concepts and themes than by running LDA alone or directly asking LMs to list topics. |
Pathway to Relevance: How Cross-Encoders Implement a Semantic Variant of BM25 (2025.emnlp-main)
Copied to clipboard
| Challenge: | Interpretability in information retrieval (IR) models is coarse-grained and poorly understood . a cross-encoder model extracts traditional relevance signals, such as term frequency and inverse document frequency . |
| Approach: | They analyze how a common IR model extracts traditional relevance signals . this is similar to the probabilistic ranking function BM25 . |
| Outcome: | The proposed model extracts traditional relevance signals in early-to-middle layers, similar to BM25 . the model then combine these concepts in later layers, laying the groundwork for future interventions . |